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•  Improve  the  quality  of  group  decision  making  by 

-  (1 )  enhancing  the  ability  of  each  participant  to 
assess/evaluate  their  pool  of  disparate  information 
findings 

-  (2)  simplifying  the  process  by  which  participants 
share  uniquely  held  information 

-  (3)  improving  the  process  for  integrating  this  shared 
information  into  the  on-going  decision  process  and 

-  (4)  developing  information  “drill  down”  capabilities  so 
that  participants  can  quickly  focus  on  the  differing 
subjective  assessments  that  are  causing  lack  of 
decision  consensus. 


This  year’s  Objectives 
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•  Expand  development  of  lOBs 

•  Interface  with  EWALL  project 


Contribution  to  CKM 
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What  is  the  data  saying? 

►iterative  information  collection 
and  analysis 

►  team  shared  understanding 
development 

►  develop,  rationalize,  &  visualize 
solution  alternatives 
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What  are  the  task  parameters? 

.  individual  mental  model 
construction 

•  knowledge  interoperability 
development 


COLLABORATION  AND  KNOWLEDGE  MANAGEMENT  {CKM)  PROGRAM 

TUR.\L  ^JODEL  OF  TEAI\[  C’t  JLLABORATION 
(MAlCRO-<  OCNITHT  PRCK  ESS  FOC  OS) 
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What  is  the  group  decision? 

team  negotiation  of  solution 
alternatives 

►  team  pattern  recognition 

►  team  shared  understanding 
development 

•  convergence  of  individual 
mental  models 
•sharing  hidden  knowledge 


Sample  Decision  Making  Task 
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We  have  spent  a  lot  of  money  over  the  last 
two  years  on  improving  airport  security. 


Has  Airport  Security  significantly  improved? 

-  Review  reports  and  assign  an  overall 
effect/impact  score  to  the  results: 


NO! 

Definitely 
Not  improved 


YES! 


r 


Definitely 
has  improved 
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You  search  and  retain  10  reports 
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Airport  Security  Gets  Another  'F' 


Vulnerabilities  an 
Improvements  for 


What  GAO  Found 

Numerous  ^jovernnienl  ;uul  indie 
ihe  air  cargo  system.  These  \-ulin 
of  some  air  carriers  and  freight  f< 
freight  at  varioius  handoffs  tliat  o 
shipper  to  the  iioint  when  it  is  loj 
weaknesses  in  this  prognun  coul 

or  the  Transiiorlation  Secui 
responsibility  for  ensuring  air  ca: 
key  recommendations  and  m;md 
sinc-e  1990  by  numerous  govemn 

thf»  nir  /••.iron  infiiis:ti'\.’  rtov^iaruvrirl 


LOS  ANGELES,  Sept.  3,  2002 


(CBS)  In  January  and  February,  C 
■gDScreeners  Flunk  CBS  Test  undercoverto  test  security j 
American  airports.  We  took  lead- 
bags,  which  block  X-rays,  througl 
checkpoints. 

Steve  Elson,  who  used  to  test  ch 
securitj'forthe  Federal  Aviation 
Administration,  helped  us  with  oi 

"When  the  bag  goes  through  the 
there's  a  big  black  blob,"  says  El; 
"They're  impossible  to  miss  and 
just  continually  let  it  go." 


Screeners  could  not  clearly  see  \ 
Screeners  in  Atlanta  and  .  'n  our  carry-ons  and  should  have 
washinyton  US  airport  security  under  fire 
didn't  check  . 
hays  six  inc  I 
ayain  they  it 
this  time. 


Security  on  US  domestic  flights  i 

By  BBC  News  Online's  Sf 
Matthews 
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AIRPORT  PASSENGER  SCREENING 


Preliminary  ObEervatione  on  Progress 
Made  and  Challenges  Remaining 


Wtiol  GAO  Found 
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Secluity,  Smith’s  Way 
The  cyber  model 


rport  Security  Issues 


made  security  an  extremely  high  priority 
ound  the  world.  Although  there  has  been  an  increase  in  the  level 
the  advice  on  this  page  should  provide  an  overview  of  what  to 
ig  through  many  of  the  world's  airports. 


SPECIAL  REPORT 

THE  STERILE  AIRPORT 


Near-total  airport  security  is  possible. 
Technologies  are  in  sight  to  seal  the  leaks,  spot 
the  bad  guys,  find  the  bombs. 


US  airline  security  has  suff 
breach  in  history. 

At  least  four  separate  teai 
boarded  planes  and  hijack 
hours  of  each  other. 

But  security  experts  say 
the  terrorists  had  been 
presented  with  virtually 
an  open  goal. 

Security  on  US  domestic 
flights  is  so  relaxed  that 


By  James  D.  Miller,  assistant  professor  of  economics.  Smith  College 
October  30,  2001  9:45  a  m. 

Unleash  the  marketplace  to  strengthen  airport  security.  The 
socialistic  solution  of  federalizing  airport- security  workers  will  deny 
us  the  creativity  we  need  to  fight  terrorists.  Only  by  utilizing  the 
constant  competition  that  the  free  market  provides  can  we  protect 
America's  skies. 

Computer  networks  have  to  endure  incessant  attacks  from  hackers. 
Network  providers  have  to  continually  strengthen  their  defenses  to 
ward  off  new  types  of  assaults.  Hackers  have  consequently 
increased  computer  security  and  have  made  the  U.S.  more  resistant  to 
terrorist  cyber- attacks.  Computer  security  is  not  provided  by  the 
government,  but  rather  by  a  marketplace  that  punishes  any  firm  that 
can't  protect  its  electronic  assets.  America's  airplanes  should  be 
protected  by  a  similar  free  market  approach. 


y  Aiithority  (< 
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AIRPORT  SECURITY  FORTHE  9/11  AGE 

Recent  dummy-weapon  tests  at  airports  show  it’s 
still  possible  to  get  guns  through  security,  here's 
how  a  super-secure  airport  would  work. 


save  this  email  this  print  this  most  popular 


We  asked  Isotec  Inc.,  a  Denver-based  security 
systems  design  firm,  to  help  us  engineer  an 
airport  that  would  target  terrorists  without 
gumming  up  passenger  traffic.  We  also  sought 
input  from  CompuDyne  Corp.,  Viisage 
Technology,  General  Defense  Systems,  and  other 
companies  that  make  and  install  security 
equipment.  In  this  exercise,  money  was  no 
object;  safety  was  our  only  concern. 

We  set  a  target  date  of  five  years  from  now.  But 
much  of  the  technology  is  available,  or  will  be 
very  soon.  The  goal;  Every  person,  every  bag, 
and  all  supplies  and  equipment  in  an  airport  will 
be  tagged,  tracked,  and  instantly  locatable. 

1.  CHECK-IN  AND  SECURE  ID 


Initial  safeguards  appear  a  mile  outside  the 
airport.  Scanners  at  tollbooth-like  structures 


Subjective  conclusions  from 
each  of  the  reports 
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Vulnerabilities  and  Potential 

Improvements  for  the  Air  Cargo  Sy  Airport  Securrry  Gets  Another  F 
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...but  other  subjective  estimates  have  also 
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Source 

Credibility? 


DCODE  Solution:  Convert  IMPLICIT 
subjective  estimates  into  EXPLICIT  estimates. 
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SPECIAL  REPORT 

THE  STERILE  AIRPORT 

Near-total  airport  security  is  possible. 
Technologies  are  in  sight  to  seal  the  leaks,  spot 
the  bad  guys,  find  the  bombs. 

by  Dan  Tynan 
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AIRPORT  SECURITY  FORTHE  9/11  AGE 

Recent  dummy-weapon  tests  at  airports  show  it's 
still  possible  to  get  guns  through  security,  here's 
how  a  super-secure  airport  would  work. 


We  asked  Isotec  Inc.,  a  Denver-based  security 
systems  design  firm,  to  help  us  engineer  an 
airport  that  would  target  terrorists  without 
gumming  up  passenger  traffic.  We  also  sought 
input  from  CompuDyne  Corp.,  Viisage 
Technology,  General  Defense  Systems,  and  other 
companies  that  make  and  install  security 
equipment.  In  this  exercise,  money  was  no 
object;  safety  was  our  only  concern. 

I  We  set  a  target  date  of  five  years  from  now.  But 
much  of  the  technology  is  available,  or  will  be 
very  soon.  The  goal;  Every  person,  every  bag, 

- ^  and  all  supplies  and  equipment  in  an  airport  will 

H  be  tagged,  tracked,  and  instantly  locatable. 
save  this  email  this  print  this  most  popular  ■■ 

*  1.  CHECK-IN  AND  SECURE  ID 

Initial  safeguards  appear  a  mile  outside  the 
airport.  Scanners  at  tollbooth-like  structures 

ainnn  rnj»H«  aim  their  lacer  crannerc  at 


Credibility? 


Encapsulate  the  scores  into  an  icon  (caiied  an  information  Object, 
iOB)  that  dispiays  information  quaiity,  impact  and  importance 


informationlimportance: 
Size  of  coior  bar  (1 ,  2  or 
3  sections  fiiied) 


Effect/impact:  Coior 


Quaiity  of  information 


iOB 


Another  look  at  the  10  reports 

IMPORTANCE 

Drop  all  items  with  beiow  average 
Source  Credibility. 


Very 

High 


BBC  NEWS 


ADlVilMM2 

#68  CRED  TJME  CDNF 


Average 


BARCHART 

SMITH ’S  WAY 

ADlyiN  M2 

#68  CRED  TIME  CONF 

AD|yllMM2 

#68  CRED  TIME  CONF 

^High 


Low 


Source  Credibility 
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Documents  vs 


lOBs 


JSRfmR 

Syst&ms 
San  Diego 


□CODE  lOB  Process 


What  are  the  key  Essential  Elements  that 
need  to  be  Abstracted  from  an  Information  Item? 


San  Diego 


Where  does  it  Fit? 


-  i.e.  which  decision  criteria/factor  (e.g.  cost,  risk,  etc.)? 


How  good  is  the  information  [Quality 

-  What  is  the  Credibility  of  the  source? 

-  How  Timely  is  this  information? 

-  How  much  Confidence  do  I  have  in  the  information? 

What  is  the 
criterion? 

-  Positive  or  Negative? 

-  Strong  or  Weak? _ 

What  is  the  Importance  bf  this  item  relative  to 
other  items? 


We  access  these  subjective  assessments 
via  an  Abstraction  Template  Structure 
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We  display/share  these  assessments  using 
an  Information  Object  (lOB) 
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Busiiie&^\^k  online 


Title: 

Criterion: 

Credibility: 


c  hist  r  admn  r  gain  c  pros 

V.Low/Poor  A>vg.  V.  High/Good 

r  I  r  I  r  K|  v  t  T| 


1 :  Select  document  for  retention 


Title: 
Criterion: 

Credibility: 

Timeliness: 

Confidence: 

Effect: 

Importance: 


[growth 


C  HIST  C  admn  GAIN  C  PROB 

V.Low/Pcior  Airg.  V.High/Good 


(T 


9  C  9 


9  C  9 


9  C  9 


V.Neg. 


Neutral 


V.Pos. 


9  C  9 


9  r  9 


Discard  C1)Airg.  C2)High  C3)^/High 


Submit 


Reset 


Close 


Step  2:  Activate  Abstraction  Template 


Step  3:  Complete  Assessment 


□CODE  Software  creates  and 
stores  an  Information  Object 
(lOB)  in  designated  database 


growth 


GAIN  M2 
#32 


CRED  TIME  CONF 


Information  Objects  (lOBs) 


The  creation,  use  and  sharing  of 
Information  Objects  (lOBs)  is 
the  key  critical  concept 

in  DCODE. 
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Information  Object  (lOB) 
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An  lOB  is  an  iconic  encapsulation 
of  the  subjective  assessments  an 
individual  has  assigned  to  a 
particular  information  item. 


It  is  automatically  created  from  the  completed 
abstraction  template. 

It  is  analogous  to  military  tactical  symbology. 

lOB  Representation 

Original  of  subjective  assessments 

document 


'Hk  CjIkt  Wnliif  UitErr  Ijidiihccif]'  Soul  FApo^cd 


SRfmR 

lOB  Encapsulation  Process  3- 


Title: 
Criterion: 

Credibility: 

Timeliness: 

Confidence: 

Effect: 

Importance: 


4 

[growth 


T 


[growth 

c  hist  C  aDMN  GAIN  C  PROB 

V.Low/Poor  A>rg.  V.  High/Good 


(T 


V.Heg. 


Neutral 


r 

r  1 

1  ^ 

^  1 

DiscaM  (2)High  C3>/.High 


Submit 


Reset 


t 


OlQse 


Importance: 

Discard 


Avg. 
High 
V.  High 


Hyperlinked  to  doc. 


Who  originated  the  lOB 


Unique  identifier 


Effect/Importance  Coding 
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IMPORTANCE 


EFFECT  ON  CRITERION  IS  COLOR-CODED 


size«  SiZ6«  SiZG 

Very  Negative 

Negative 

Positive 

Very  Positive 

Average 

Very  High 


Importance 
“Power  bar” 


2  Squares 


Document  is  of  High 
Importance  and  has 
a  Very  Positive  effect 
on  the  Gain  Criterion 


De-Cluttering  the  Display 
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Minimal  View  (keyword,  effect,  importance) 


Recaiis 


Analysis  FROM  The  Stock  Consultant 


1997 

Growth  Prosnects 

VectorVest  Analysis 

Salsitas 

TreviPark  System 

\ 

irior  contract  2^1 

Which  Item?  The  Effect?  The  Importance? 


Full  View  (ail  parameters)  available  as  a  drill-down  capability 


Recaiis 


HIST  M2  #3 


CRED  TIME  COHF 


Analysis  FROM  The  Stock  Consultant 


HIST  M2  #12 


CRED  TIME  COHF 


prior  contract 


HIST  M2  #25 


CRED  TIME  COHF 


Growth  Prospects 


HIST  M2  #10 


CRED  TIME  COHF 


TreviPark  System 


HIST  M2  #16 


CRED  TIME  COHF 


VectorVest  Analysis 


HIST  M2  #11 


CRED  TIME  COHF 


prior  contract  2^1 


HIST  M2  #22 


CRED  TIME  COHF 


Why  Use  lOBs? 
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•  lOBs  improves  an  individuars  decision  making  ability  by 
simplifying  the  evaluation  of  disparate  information  sources. 


Sharing  and  Integration 
of  Information 


V 

Syst&ms  Center 
San  D/ego 


lOB  Sorting/Analysis  Techniques 


Participanfs 

Individual 

Display 


HIST 

ADMN 

GAIN 

PROS 

Overall  Decision  Recommendation: 


[Clear  Ratings] 


Give  me  non-redundant  lOBs 
from  other  group  members 


1997 

Salsitas 

VectorVest  Analysis 

f 

irior  contract  2^1 

Growth  Prosnects 

TreviPark  System 

fUSMUMI? 


|DCODE  Provides  “one-click”  sharing 
of  Uniquely  Held  Information 

Information  that  I  have: 


Analysis  FROM  The  Stock  Consultant 

Salsitas 

VectorVest  Analysis 

Recalls 

Growth  Prosnects 

I 

irior  contract 

1997 

TreviPark  System 

1 

)rior  contract  2^1 

“Add  Group” 


Adds  three  new  group  lOBs 


Analysis  FROM  The  Stock  Consultant  Salsitas 


Growth  Prospects 


Globalization 


VectorVest  Analysis 


prior  contract 


Recalls 


1997 


t 


TreviPark  System 


globalization 


prior  contract  2^1 


t 


DCODE  lOBs  Improve  the 
Integration  of  Shared  Information 


miTy 

w 

^  Diego 


^JC] 

^  ^  ^  ^  ^Iwih  jjfP-brUd  ^MBiiy  ^  S  ^  ^  'J 

c«  fMPtw  T5* 

fCI  http  Mht«lri':f'  (C'rjVKilHi'lX-ir.JPWlI?^  h^T. 

d  f?4*  J  E 

i-i 

j  ^  ;a 

- 3 

Informatloi 


vs.  PfAVY  A\ 
W.\Hh  AIIK  SI 


Member  already  has  a 
high  cognitive  burden. 
What  is  the  likelihood 
these  will  be  integrated 
into  his  decision  process? 


I  4.FinTwn  rh*^i.p  f,Kt$  diFin^KP  ih.  fpt 

I  1«h-  f  p  f 

■  crWfHl-  iPt'fCq  rtVfIra  M  i  111  1ii Jiili  iii  i  ^  Vn  J  ■ 

I  |tpwT.UHM»id  Mini  lid  ih.4T.i4^di<4puti 

■  TV -uifTWl- PT^«  figKTP  Oi; 


“Add 


Administration 


r 

r 

r 

Submit  1 

Overall  Criterion  Score:  i 

— — 1 

Industiy  Projects  Index 


TPS  Reports 


Red  Tape 


LMT 


ADMN  M2 
#17 


CRED 


TIME 


ADMN  M2  #1 


CONF 


CRED  TIME 


CONF 


ADMN  M2 
#13 


consent  agree 


Memo  1 


ADMN  M2 
#29 


CRED 


ADMN  M2  #2  ' 


CONF 


CRED 


TIME 


V  2" ' 

CONF 


ADMN  M2 
#18 


CRED  TIME  CONF 


Add  Group 


Domestic  Shipping 


#26  CRED  TIME  CONF 


Globalization 


#37  CRED  TIME  CONF 


Key  Concepts  in  lOB  Application  to 
Shared  Understanding 
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J  3  filJNKiL  J<Wr  J  H  "  J  ti 


~3  ^  ,!  .£]  ^ 


Busuvcs^fek 


kHMWtdkf  MJjlIJMtIllTM  l^jpywfalltLt 


!*■  ii^  ■ 


PTipupt-TtHimim^M^iitVi^TAHi  iM<iiP  i  iwii  ni  Iw  MipMKjirt  Bn<  N  *4* 


[CB  pi^  J 


INFORMATION 

SHARING 


INFORMATION  ABSTRACTION 


Title: 

Criterion: 


[ 


Credibility: 
Timeliness: 
Confidence: 

Effect:  I 

Importance: 

Submit  I  Reset  I 


V.Heg. 


T  r  admn  c 

Aurg. 

GAIN  C  PROB 

V.High/Gocd 

1  ^  1 

1  ^ 

1  r 

1  <'  1 

1  - 

1  ''  ?i 

HeLitr^l 

V.Pos. 

r 

r 

r  ^  ^ 

Discard  (I^Airg.  (2)High  (3)^/.High 


Close 


INFORMATION 

ENCAPSULATION 


growth 


V 


GAIN  M2 
#32 


CRED  TIME  CONF 


New  Concepts  (not  in  demo) 
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•  Entering  personal  knowledge 

•  Encoding  Quality  of  Information 


The  “EQI”  bar 


JSRfmR 

Two  Information  Sources: 


I  Relief  Web] 


ReUefWe^ 

FWebI 


Source:  British  Red  Cros”. 
Date:  14  Feb  2001 

Red  Cross  resc 

Red  Cross  emergency  tear 
Salvador  and  pulled  atleai 
Salvadorean  Red  Cross  an 
patients  evacuated  from  fc 

The  earthquake  measured 
Salvador.  According  to  Sa 
Occurring  just  one  month 
than  half  a  million  homelei 
affected  area. 


Some  300  Red  Cross  volu 
victims.  Within  the  first  fel 
distributed  rolls  of  plastic  | 


Title: 

Criterion: 

Credibility: 

Timeliness: 

Confidence: 


j 

Ready  to  respond  at  short  notice 


Hjelpekorpset  [Red  Cross  Rescue  Corps]  is  Norway’s  largest  volunteer 
rescue  preparedness  organisation.  It's  main  objective  is  to  assist  at  th 
scene  of  an  accident  both  during  and  after  natural  disasters  and  to  pro 
other  humanitarian  relief. 


Federation  teams  were  dis- 


"The  water  supply  system 
damaged.  It  is  essential  fo: 
population",  saysIainLoi 


Delegates  were  sfiUinthe 
pre  s  ent  in  the  worst  affe  c1 
Vincente,  and  medical  te 
Red  Crescent  appeal  will 
As  the  four  district  hospitl 
teams  were  sent  to  assist  hi, 


ntH 


Hjelpekorpset  ams  to  train  its  members  to  be  prepared  and  ready  to  resi 
effectively  and  to  spread  awareness  of  general  first  aid.  Hjelpekorpset  a\r 
be  a  quality  leader  within  the  relevant  areas;  by  caring  and  taking  respons 
and  initiative  for  people  who  need  help  and  compassion. 

Today.  Hjelpekorpset  has  more  than  1 5,000  volunteers,  aged  1 7  and  ove 
Hjelpekorpset \s  an  extremely  active  organisation  and  each  member  spe 
considerable  number  of  hours  yearly  educating  themselves  and  carrying 
rescue  work.  There  is  a  rescue  corps  on  emergency  standby  at  335  locat 
across  the  country. 


Effect  SEAT 
Importance: 


Save/Share 


I  [pk] 

HIST  ADMN  GAIN  C  PROS 

V.Low/Poor  A>rg.  V.  High/Good 


V.Neg. 


Neutral 


V.Pos. 


DiscarxJ  C2)High  C3)V.High 


Save 


Res  et 


Close 


(1)  Internet,  Documents,  Databases,  etc 


...and  (2)  Personal  Knowledge 
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Title:  | 

Criterion: 

C  HIST 

V.Low/Poor 

Credibility:  | 

Timeliness:  | 

Confidence:  | 

V.Neg. 

Effect  SEAT] 

Importance: 

1 

Save/Share 

Save 

C  HIST  ADMN  C  gain  PRQB 


PK 


A/g. 


Neutral 


Reset 


V.  High/Good 


Discard  (1)Afg.  (2;iHigh  (3)W.High 


Close 


A  click  here 
produces  a  text  box 
for  information  entry: 


Created  by:  M1  BILL 
On:  9/23/03  1705 

In  1998  I  was  involved  in  a 
Simulated  NEO,  and  we 
found  that.... 

Save/Finish 


Information  Coding 
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•  Three  most  critical  elements  are: 
•  Effect/Impact :  Color  (4  or  5  levels) 


•  Importance:  Size  (3  levels) 


•  Quality:  Composite  of  several  parameters 

Color  Pattern  (2  levels) 

Quality  is 
Beiow 
Average 


Quality  is 
Average 
or  Better 


Coding  Quality  of  Information 
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Quality  is  a  composite  score: 


Decision  Team  sets  up  a  scoring 
aigorithm,  e.g. 

“fiag  any  totai  score  5  or  iess” 


Timeliness 

Credibility 

Confidence 


0  12  3 


Timeliness 

Credibility 

Confidence 


Timeliness 

Credibility 

Confidence 


1  2  3 


Quaiity  is 
Average 
Or  better 


Total  Score  =  6 


Quaiity  is 

Beiow 

Average 


Total  Score  =  4 


gpg,  ggg. 
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The  Effect/Quality/Importance  (EQI)  Bar 
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AVERAGE 

IMPORTANCE 

Good  ? 


Very  Pos. 
Pos. 
Neg. 
V.  Neg. 


HIGH 

IMPORTANCE  Quality 

Good  ? 


Very  Pos. 
Pos. 
Neg. 
V.  Neg. 


VERY  HIGH 


Quality 

Good  ? 


IMPORTANCE 
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Do  lOBs  really  help? 


PG  School  Experiment 
18  Officers  (Summer  03) 

•  Display:  Text  vs  lOBs 
Decision:  Positive  vs  Negative 

Display 


JSFmm 

V 
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San  Dfego 


Positive 

Decision 

Negative 


Text  Only 

lOBs 

N=5 

N=5 

N=4 

N=4 

N=9 

N=9 

N=10 


N=8 


N  (tot)=18 


Task: 
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•  Should  we  use  Islandia  as  the  refugee  site? 
•  Sees  30  information  items  (randomized) 

-5  decision  criteria 
•  6  items  per  criteria 


The  Five  Decision  Criteria 
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[COMmunications  -  TSPortation  -  LABor-  ADMistration  -  SANitation  ] 

•  Communications  Facilities  (COM) :  As  s  e  s  s  th  e  c  o  m  m  u  n  i  c  ati  o  n  fa  c  i  I  iti  e  s  th  at 
are  available  in  Islandia,  including  land  telephone  systems,  radio,  TV,  cellular 
phone  availability  and  coverage,  etc. 

•  T ransportation  Facilities  (TSP) :  As  s  e  s  s  th  e  tra  n  s  p  o  rtati  o  n  fa  c  i  I  iti  e  s  th  at  a  re 
available  in  Islandia,  including  roads,  docks,  airports,  etc. 

•  Labor  Pool  (LAB):  Assess  the  labor  pool  that  would  be  available  to  staff  the 
camp  in  Islandia,  including  size  of  the  pool,  quality  of  workers,  work 
ethic/tradition,  etc. 

•  Administrative  Ftequirements  (ADM);  Assess  the  administrative  requirements 
needed  to  set  up  the  camp  in  Islandia,  including,  permits,  fees,  environmental 
considerations,  bureaucratic  red  tape,  graft,  bribes,  etc. 

•  Sanitation^ealth)Medical  conditions  (SAN);  Assess  the  sanitation,  health  and 
medical  conditions  expected  in  Islandia,  including  drinking  water,  sewerage 
disposal,  medical  facilities,  infectious  diseases,  etc. 


Text  Condition  (9  subjects) 
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1/30 

....A  cultural  tradition  in  Islandia  is  that  each  worker  is  given  a  single  two  month 
vacation  each  year,  which  he  can  take  anytime  during  the  year.  All  the  employee  has  to 
do  is  give  the  employer  a  one  week  notice  before  going  on  vacation.  This  has  caused 
unexpected  and  disrupted  work  shortages  when  several  employees  elect  to  take  the 
vacation  at  the  same  time.... 

< 

Criterion: 


Read  this,  then  assign 
it  to  one  of  the  five 
criteria 

(do  30  of  these) 


[COMmunications  -  TSPortation  -  LABor-  ADMistration  -  SANitation  ] 

•  Communications  Facilities  (COM):  Assess  the  communication  facilities  that 
are  available  in  Islandia,  including  land  telephone  systems,  radio,  TV,  cellular 
phone  availability  and  coverage,  etc. 

•  Transportation  Facilities  (TSP):  Assess  the  transportation  facilities  that  are 
available  in  Islandia,  including  roads,  docks,  airports,  etc. 

•  Labor  Pool  (LAB):  Assess  the  labor  pool  that  would  be  available  to  staff  the 
camp  in  Islandia,  including  size  of  the  pool,  quality  of  workers,  work 
ethic/tradition,  etc. 

•  Administrative  Requirements  (ADM):  Assess  the  administrative  requirements 
needed  to  set  up  the  camp  in  Islandia,  including,  permits,  fees,  environmental 
considerations,  bureaucratic  red  tape,  graft,  bribes,  etc. 

•  Sanitation/Health/Medical  conditions  (SAN):  Assess  the  sanitation,  health  and 
medical  conditions  expected  in  Islandia,  including  drinking  water,  sewerage 
disposal,  medical  facilities,  infectious  diseases,  etc. 


lOB  Condition 


(9  subjects) 
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1/30 

1 

The  environmental  protection  effort  in  Islandia  is  poorly  organized  hand  has  had  littk 
effect.  Environmental  impact  is  not  a  consideration  in  any  awarding  of  construction 
permits. 

<C 

. - 1 

Read  this,  then: 


Title: 

Criterion: 

Effect: 


Submit  >  I 


Assign  a  keyword 
Assign  it  to  a  criterion 
Evaluate  its  effect  on 
the  criterion 


[COMmunications  -  TSPortation  -  LABor-  ADMistration  -  SANitation  ] 

•  Communications  Facilities  (COM):  Assess  the  communication  facilities  that 
are  available  in  Islandia,  including  land  telephone  systems,  radio,  TV,  cellular 
phone  availability  and  coverage,  etc. 

•  Transportation  Facilities  (TSP):  Assess  the  transportation  facilities  that  are 
available  in  Islandia,  including  roads,  docks,  airports,  etc. 

•  Labor  Pool  (LAB):  Assess  the  labor  pool  that  would  be  available  to  staff  the 
camp  in  Islandia,  including  size  of  the  pool,  quality  of  workers,  work 
ethic/tradition,  etc. 

•  Administrative  Requirements  (ADM):  Assess  the  administrative  requirements 
needed  to  set  up  the  camp  in  Islandia,  including,  permits,  fees,  environmental 
considerations,  bureaucratic  red  tape,  graft,  bribes,  etc. 

•  Sanitation^ealthJUIedical  conditions  (SAN):  Assess  the  sanitation,  health  and 
medical  conditions  expected  in  Islandia,  including  drinking  water,  sewerage 
disposal,  medical  facilities,  infectious  diseases,  etc. 


JSRfmR 

Completed  Item  (lOB)  X 


3/30 


"....The  major  ports  of  Islandia  are  less  than  three  days  sailing  from  the  primary 
disaster  recovery  supply  center  at  Naha.  Naha  also  has  three  supply  ships  that  are 
permanentiy  ioaded  to  be  used  in  quick  response  assistance..." 


Completed  lOB 
(does  30  of  these) 


[COMmunications  -  TSPortation  -  LABor-  ADMistration  -  SANitation  ] 

•  Communications  Facilities  (COM):  Assess  the  communication  faciiities  that 
are  availabie  in  isiandia,  inciuding  iand  telephone  systems,  radio,  TV,  ceiiuiar 
phone  avaiiability  and  coverage,  etc. 

•  Transportation  Facilities  (TSP):  Assess  the  transportation  facilities  that  are 
avaiiabie  in  Isiandia,  including  roads,  docks,  airports,  etc. 

•  Labor  Pool  (LAB):  Assess  the  iabor  pooi  that  would  be  available  to  staff  the 
camp  in  Isiandia,  including  size  of  the  pooi,  quality  of  workers,  work 
ethic/tradition,  etc. 

•  Administrative  Requirements  (ADM):  Assess  the  administrative  requirements 
needed  to  set  up  the  camp  in  isiandia,  inciuding,  permits,  fees,  environmental 
considerations,  bureaucratic  red  tape,  graft,  bribes,  etc. 

•  Sanitation/Health/Medical  conditions  (SAN):  Assess  the  sanitation,  heaith  and 
medicai  conditions  expected  in  Isiandia,  inciuding  drinking  water,  sewerage 
disposai,  medical  facilities,  infectious  diseases,  etc. 


Decision  Tasks: 
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Hffl  “w 

COM 
TSP- 
luAf! 

3AH- 

OweraA  Rwornmernl  Aon: 

r  r 

Sidamit  J  Reset  j 


5  Criteria  Decisions, 
1  Overali  Decision 


Decision  Display:  Text 
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viswiIaT 


"3  I 


Communications 


T|i$  frwwm9»ii?i  p^tiwpeflprt'r!  iei?ndifl‘5  posnif  prgwitrt  hits  his  HU!? 

rHiLa  coridideralithi  ig  ai^aiMiaidEni^areoii^liuiEiOM  [Mr^ 


A  BlmL/a1lDns4?fcl5@  W\  Ielan4^  EihD^BdVi^d~iB  Eme?'9erH:^M€dl[Bl 

virr  madaqualD  in  Lvim*  4f  hancfing  amfthn;|  otliD  Ihjfn  Iht  sniilhil  disailer  tilualii 


.  The  majof  ports  of  lalemdie  ara  lesa  dun  three  da^s  sailli^  Trorh  tte  primary  dlsaslet 

rg?Dfl??rf  ¥upplV  center  5I  W?n?  hg®  fhrgg-  ®uppty  ®hipf  tn«  flrig  pgnm^ngfilhr  Insds^ 

to  Peuaed  Ir^duick  reaporraa  aasialanr-a. .' 


E¥firY  emaitoivn  m  Isiandia  has  ^flieael  one  phpre 


.  The  rtiai^  air  rQuIc-Da  Isl^dii  it  Tiurn  Chinr^E!  ThairnulB,  K^wv^i,  requires  lhalwc  IVinMr 
the  r^a1lanoTC-arht3*sl,  at>d  OarhtHellias  denied  permlesnrilD  tty oror their  airspace. The  t*a 
qpbpiii  arfelD  mddrfKtht-niglil  plln  h  Id  usif  andirifei  [Hp«rtijiE  airporl  9Dlhd'1tit‘tt  dplionl 
resulte  In  e4(^ess>¥ely  long  nighle,  llmhe  pargo  capaertv,  and  ^Mee  die  dafly  arhourk  of 
®Mpplli~s  IhgA  cam  Inn^poHjid  1d  Islendia  *' 


Owera*  RecommerHlsdon: 


HP"! 

^  D 

ReaeC  j 

The  items  the  subject 
assigned  to  the 
Communications 
criterion 


Decision  Display:  lOB 
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\^ry  Heg. 


\^ry  Pos. 


Submit 


Reset 


Click  on  the  hyperlinked  titles  in  the  lOB  to 
display  the  original  tejct  here. 


Hyperlinked 


Positive  vs  Negative 
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Vt.  of  Subjects  should  make 
a  decision  that  is  Positive: 

3  of  5  criteria  are  Positive 


72  of  Subjects  shouid  make 
a  decision  that  is  Negative: 

3  of  5  criteria  are  Negative 


Positive  Criterion:  4  of  the  6  statements  are  positive 
Negative  Criterion:  4  of  the  6  statements  are  negative 


lOB  Subjective  Assessment 
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One  Last  Request:  Please  select  one  of  the  options  below  in  terms  of  how  usefui  the  iOBs 
were  in  making  your  scoring  decision  (this  wouid  be  as  versus  just  seeing  the  text  iisting 
of  the  items  you  assigned  to  each  criterion) 


Somewhat 

distracting 

No  Effect 

Heiped 

Somewhat 

Heiped  a  lot 

r 

r 

r 

r 

Submit 
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RESULTS 


Time  Factor 
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25 


Classify  the  30  Make  the  6  decisions 

lOB  subjects  took  an  average  18  seconds  longer 
per  item  to  enter  keyword  and  make  evaluation 


TIME  TO  MAKE  DECISIONS 
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M 

I 

N 

U 

T 

E 

S 


Participants  took  significantly  longer  to  complete  the  decision 
process  when  the  predominance  of  information  was  positive. 


20 

18 

16 

14 

12 

10 

8 

6 

4 

2 

0 


Errors  in  Criterion  Scoring 


45  Max 

1 

\ 

I 

- \ - 

- \ - 

— , — 1 — 1 — , 

- - ^ ^ ^ - - - 

Text  lOB  Text  lOB 

(includes  (includes  (Reversals  (Reversals 

neutral)  neutral)  only)  Only) 
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ERRORS  OVERALL 
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10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 


9  Max 

- \ - 

- 1 - 

- \ - - - 1 

Text 

(includes 

neutral) 


lOB  Text  lOB 

(includes  (Reversals  (Reversals 

neutral)  only)  Only) 


Overall  Ratings 
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Overall  Final  Ratings 


□  Text 

6/6  errors 

2/2  errors 

1/1  errors 

- ^ — 

- ^ - 

- ^ ^ - 1 

Neg  Neutral  Pos 

0/9  Correct  Decisions! 


Overall  Final  Ratings 


1 

■  lOB  1/5  errors 

2/2  errors 

0/2 

■  1 

- ^ — 

1,1.1 

L _ , 

■ 


Neg  Neutrai  Pos 

6/9  Correct  Decisions! 


The  Two  Display  Conditions: 
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TEXT 


lOB 


0  out  of  9  Correct  Overall  Decisions 


6  out  of  9  Correct  Overall  Decisions 


lOB  Subjective  Assessments 
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7 

6 

5 

4 

3 

2 

1 

0 


□  Max  =  9 

- ^ ^ - 

- ^ - 

- 1 

lOBs  HELP? 


Distract  No  Effect  Somewhat  A  Lot 


Research  Summary: 
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•  Decision  time  same  for  Text  and  lOB. 

•  Decision  time  longer  for  positive  information. 

•  Text  participants  make  more  reversal  errors  in 
scoring  individual  criteria. 

•  In  making  Overall  decisions: 

-  Text  more  likely  to  use  Neutral  rating. 

-  Text  more  likely  to  make  reversal  errors. 

-  lOB  make  more  accurate  decisions. 

•  lOB  participants  feel  lOBs  are  helpful/useful. 


Hidden  Profile  Experiment 
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•  Subjects  saw  12  information  items,  leading 
to  a  Positive  or  Negative  recommendation 

•  Then  saw  10  more  items  that  should 
cause  a  reversal  in  the  recommendation 

•  Out  of  20  eligible  subjects,  only  7  (35%) 
detected  the  hidden  profile. 

•  Shifting  from  Negative  to  Positive  was 
significantly  more  difficult  (1  out  of  6) 
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Expected  Final  Products 
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•  A  shrink-wrapped  software  package: 

-  Runs  in  the  background  on  a  WAN  or  LAN 

-  Set-up  by  the  assigned  decision-task  group 

-  User-designed  abstraction  template 

-  lOB  Generation  algorithm 
-Optimized  sorting,  analysis  package 

-  Group  interactive  lOB  display  area 

-  Uses  existing  COTS  audio/visual  collaborative 
system  to  reduce  conflict,  reach  consensus. 


Possible  Application  areas 
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•  Proposal  submitted  to  Home  Land  Security: 

-  “Information  Objects  (lOBs):  A  Knowledge 
Management  Tool  for  Information  Fusion,  Correlation 
And  Distribution” 

•  Proposal  submitted  to  DDR&E 

-  “Iconic  Abstractions  of  Subjective  Knowledge  for 
Asynchronous  Collaboration” 

•  DCODE  briefed  to  Counter  Drug  Investigation 
Center  (CIC) 


Recent/Planned  Publications 
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•  Information  Exchange  and  Display  in  Asynchronous  C2  Group  Decision 
Making.  Paper  presented  at  8th  International  Command  and  Control  Research 
and  Technology  Symposium,  Washington  D.C.,  June  2003. 

•  The  Effects  of  New  Information  on  Decision  Modification:  The  Roie  of  Display 

Format  and  Direction  of  Impact.  SSC-SD  Technical  Report,  in  prep,  Jan  04. 

•  Improving  Individual  and  Team  Decisions  Using  Iconic  Abstractions  of 

Subjective  Knowiedge.  Paper  submitted  to  9th  International  Command  and 
Control  Research  and  Technology  Symposium,  San  Diego,  June  2004. 

•  An  experimental  comparison  using  text  documents  versus  iconic 
representations  in  complex  decision  making.  SSC-SD  Technical  Report,  in 
prep.,  Jan  04 

•  Essential  information  characteristic  needed  for  any  retained  decision-relevant 

information  item.  SSC-SD  Technical  Report,  in-progress. 


lOB  On-Line  Demo 
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Available  at  two  sites: 

http://64.66.5.34/demo/ 


http://www.ocf.berkelev.edu/~slinlee/dcode/demo/ 
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Back-up  slides 
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Scoring  Assignments  by  lOB 
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Research  Questions 
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•  What  are  the  critical  subjective  information  assessments 
(parameters)  that  each  individual  assigns  to  a  retained 
information  item? 

•  How  can  we  quantitatively  capture  these  assessments 
without  placing  an  undue  cognitive  burden  on  the 
individual? 

•  How  do  we  best  display  this  captured  information  to 
improve  the  ability  of  the  individual  to  make  on  overall 
assessment  of  his  information  pool? 

•  How  do  we  display,  share  and  transfer  this  captured 
information  to  ensure  that  the  group  sees  and  uses  to 
the  entire  pool  of  relevant  information  to  reach 
consensus  on  the  final  overall  group  recommendation? 


Decision  Making  Requirements 
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•  Once  a  group  has  been  assigned  a  decision 
making  task,  they  need  to  determine  two  critical 
issues: 

-  What  are  the  decision  criteria  (factors)  that  we  will  use? 

•  e.g.  Cost,  Risk,  Duration,  etc. 

-  How  many  options  (COAs)  are  we  considering? 

•  One  (Yes/No,  Go/No  Go) 

-  Should  we  do  A? 

•  Multiple  COAs 

-  Should  we  do  A  or  B  or  C  or ....? 


Template  changes  only 
slightly  from  task  to  task 
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Different  Decision  Tasks  modify  only  2  parameters  of  the  Abstraction  Template: 
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Airport  Security  Gets  Another  T' 


LOS  ANGELES,  Sept.  3,  2002 


Steve  Elson,  who  used  to  test  checkpoint 
securitji'forthe  Federal  Aviation 
Administration,  helped  us  with  our  tests. 


(H>  Screeners  Flunk  CBS  Test 


(CBS)  In  January  and  February,  CBS  News 
went  undercover  to  test  security  at  major 
American  airports.  We  took  lead-lined  film 
bags,  which  block  X-rays,  through 
checkpoints. 


(Photo:  CBS/AP) 


"When  the  bag  goes  through  the  X-ray, 
there's  a  big  black  blob,"  says  Elson. 
"They're  impossible  to  miss  and  yet  they 
just  continually  let  it  go" 


Screeners  in  Atl<int<i  and 
WasliiiHjton's  Reagan  N<Ttional 
didn't  check  any  of  our  film 
hags  six  months  ago.  and 
again  they  missed  all  of  them 
this  time. 


Screeners  could  not  clearly  see  what  was 
in  our  carry-ons  and  should  have 
searched  them,  because  a  weapon  could 
have  been  hidden  in  or  under  the  film 
bags. 

But  70  percent  of  screeners  failed  to 
check  or  even  detect  the  film  bags.  At  the 
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Seciuity,  Smith’s  Way 

The  cyber  modeL 

By  James  D.  Miller,  assistant  professor  of  economics.  Smith  College 
October  30, 2001  9:45  am. 

C^nleash  the  marketplace  to  strengthen  airport  security.  The 
socialistic  solution  of  federalizing  airport- security  workers  will  deny 
us  the  creativity  we  need  to  fight  terrorists.  Only  by  utilizing  the 
constant  competition  that  the  free  market  provides  can  we  protect 
America's  skies. 

Computer  networks  have  to  endure  incessant  attacks  from  hackers. 
Network  providers  have  to  continually  strengthen  their  defenses  to 
ward  off  new  types  of  assaults.  Hackers  have  consequently 
increased  computer  security  and  have  made  the  U.S.  more  resistant  to 
terrorist  cyber- attacks.  Computer  secrrrity  is  not  provided  by  the 
government,  but  rather  by  a  marketplace  that  punishes  any  firm  that 
can't  protect  its  electronic  assets.  America's  airplanes  should  be 
protected  by  a  similar  free  market  approach. 


This  article  is  Negative  in  supporting  the  (decision  criterion 
and  is  rated  of  High  importance.  Some  question  as  to 
Quality  of  the  information. 


This  article  is  Very  Positive  in 
supporting  the  decision  criterion  and 
is  rated  of  Very  High  importance. 
Information  Quality  is 
acceptable/good 


This  article  is  Very  Negative  in  supporting  the 
decision  criterion  and  is  rated  of  Average 
importance.  Information  Quality  is  acceptable/good 


